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Abstract 

Assessment in education employing web tools, also known as e-assessment, deals with the effective use 
of technology to support successful instruction. The aim of this study was to investigate learning 
outcomes and the students' attitudes to online Moodle quizzes in order to improve instructional design. 
The research population included 204 college students enrolled in life sciences who were participating 
in an introductory physics course. A blended learning model was used, based on large, traditional face- 
to-face lectures, practice sessions held with smaller groups of about 25 students and a rich Moodle 
learning environment. The students' knowledge and understanding were assessed weekly throughout 
the semester, using two different methods: three ordinary short written tests and online quizzes in the 
Moodle environment. The students' attitudes towards the online quizzes as compared to the written 
tests were investigated by questionnaire. Results indicate that both the average grade on written tests 
and the average grade on online quizzes were found to be significant predictors of the grade on the 
final exam. Students significantly improved their scores and greatly shortened their performance time 
on the last attempts of the online quiz, as compared to their first attempts. The investigation into the 
students’ attitudes towards online quizzes reveals a generally positive attitude. Learning outcomes and 
the students' attitudes to online Moodle quizzes were considered to improve instructional design, which 
demonstrates formative assessment in higher education. 


Keywords — Formative assessment, Higher education. Online quizzes. Physics education, Virtual 
learning environment. 
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1. Introduction and Theoretical Background 

Higher education is changing. Most if not all students now have access to learning materials via 
the Internet (Brown, 2006). Students expect and need different things from their instructors. 
They need to be guided through their learning process. They expect the instructor to teach them 
how to build their knowledge and comprehension of the given subject (Shea, Pickett & Pelz, 
2003; Clark, Kirschner & Sweller, 2012). Furthermore, they expect them to use technology to do 
this. Instructors must meet this challenge and integrate technology into their courses, effectively 
facilitating the students' learning process. The instructors must therefore be confident that the 
investment is worthwhile - that the effective use of new tools leads to better outcomes, in both 
students' attitudes and in their professional understanding. According to Trow (1999), studies are 
needed to assist teaching staffs in high schools, colleges and universities to recognize the 
possibilities of using technology as an effective tool in the teaching process and to enhance the 
positive outcomes that it may bring about. The present study falls into this category. We studied 
the implementation of Moodle quizzes in a physics college course for life science students. The 
focus was on assessing the role of the online quizzes in the learning environment in a blended 
learning model. Our investigation included three stages. We first examined the role of 
computerized assessment through online quizzes in the academic course evaluation process. In 
this stage, we investigated whether the online quizzes and the written tests used during the course 
predicted the students' scores on the final written exam. In the second stage, we focused on the 
students' behaviors and performance during the online quiz. In particular, we tried to understand 
whether many attempts on an online quiz served any benefit for the students. We examined 
possible correlations between the number of attempts and student outcomes. We believe that the 
quiz design, which combines a randomization factor and many allowed attempts, offers an 
opportunity for students to take responsibility for their own learning. The immediate feedback 
encourages the students to reflect on their understanding, to implement whatever methods are 
needed to gain more understanding, and to make another attempt to ascertain whether they have 
actually improved their knowledge. The last stage included the investigation of student attitudes 
towards the online quizzes. Another possible benefit we studied focused on the use of symbol 
manipulation when solving problems on a quiz, which encourages the students to focus on 
problem-solving strategies rather than finding a final numeric answer. The three stages are 
presented in Figure 1. 


- 189 - 



Journal of Technology and Science Education — http://dx.doi.org/10.3926/jotse.217 


1 - The feasibility of predicting 
.success in the final test of the course 
using online qmzzes 


2 - Students' behaviors and 
performance during online quiz 
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3 - Students’ attitudes towards the 
online quizzes 



Figure 1. Investigation stages 

The concept of a blended learning model refers to a learning environment that combines face-to- 
face learning sessions with self-paced, computer-assisted learning activities (Alonso, Lopez, 
Manrique & Vines, 2005). The learning environment is the entire space in which learning occurs. 
It includes all the resources (physical components, time and society) in this space, which may 
affect the learning that takes place. Since all learning is rooted in the conditions of the specific 
environment, the environment must be designed in a rich, complex and dynamic manner that 
invites interaction with different agents and fosters deep understanding. The learning 
environment should also include innovative technologies that can be used to achieve educational 
goals and improve the quality of the learning processes (Bransford, Brown & Cocking, 2000). 
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Technology has almost always been part of the teaching and learning process. The effective 
integration of technology in the classroom has been the subject of many studies, especially since 
Internet use has become so common. New terms, such as interactive learning environment (ILE), 
information and communication technologies (ICT) and blended learning have been introduced 
(Johnson, Rickel & Lester, 2000; Novak, Patterson & Gavrin, 2000; Garrison & Kanuka, 2004; 
Mor & Winters, 2007; Yen & Lee, 2011). Several studies have been conducted to investigate how 
the use of certain technologies has affected student outcomes. Belcher and Dori (2005) showed 
how the Technology-Enabled Active Learning (TEAL) project at MIT positively affected student 
outcomes in a physics course. The project improved the students' conceptual understanding, 
reduced the failure rate and improved other learning dimensions. Beatty and Gerace (2009) 
emphasized that technology alone is not enough; it must be used effectively to support successful 
pedagogy. They developed and tested a technology-enhanced pedagogy based on the classroom 
response system (CRS), also known as the clicker system. The authors' important distinction 
between technology itself and its effective use means that instructors who want to improve their 
teaching should not be satisfied with just adding technological tools to their classes. They must 
carefully plan the ways in which they can integrate these tools into their teaching. This should 
lead to the rethinking of the entire attitude towards the teaching process, encouraging more self¬ 
learning and student-centered teaching. It is not surprising, therefore, that studies have been 
written about the teaching staff: their attitudes towards integrating new technology and the need 
to prepare and guide them towards satisfactory results (e.g., Shemla & Nachmias, 2006; Pundak & 
Rozner, 2008; Pundak, Herscovitz, Shacham & Wiser-Biton, 2009; Liu, 2011). 

Design approaches in technology-enhanced learning have become popular in educational 
research. This approach is concerned with the design of learning processes, taking into account 
the complexities involved in educational settings (Wang & Hannafm, 2005; Mor & Winters, 
2007). Design-based research in education includes the systematic exploration of critical elements 
in learning environments in terms of their effect on the learning process (Collins, 1992). Kali 
(2006) used design-based methods to investigate the Design Principles Database (DPD) as a 
mechanism for supporting a community of learners who are exploring educational technologies. 
The learning environment is defined by Kali (2006) as a system that incorporates a set of 
artifacts, along with a navigation system and curriculum materials; the term 'feature' is used to 
refer to any technological artifact in the learning environment that targets advanced learning. 

This research study investigated online Moodle quizzes in order to understand the role of this 
technology component as part of the learning environment in a blended learning model. Moodle 
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(Modular Object Oriented Development Learning Environment) is a popular open-source course 
management system (CMS), also known as a learning management system (Romero, Ventura & 
Garda, 2008). Moodle was designed as a pedagogical platform to promote and assist learning in 
several dimensions (Katsamani, Retalis & Boloudakis, 2012). It offers numerous ways of 
facilitating information sharing, collaborative platforms through forums and chats, storage areas 
for course materials and student data, thus providing powerful and easy-to-use tools for active 
learning and for student assessment (Psycharis, Chalatzoglidis & Kalogiannakis, 2013). 

In our field of research, online Moodle quizzes have been used for formative assessment, which 
emphasizes the operations needed to improve student achievement in relation to desirable 
outcomes. The goal of formative assessment is to gather feedback that can be used by the 
instructor and the students to guide improvements in ongoing teaching and learning (Brown, 
Race & Smith, 1996). The integration of assessment practices embedded in instruction is critical 
for improving teaching and learning. Research implies that learning can be improved when 
teachers use instructional strategies with frequent and ongoing assessment of the students’ 
developing understanding (Treagust, Jacobowitz, Gallagher & Parker, 2001). This assessment tool 
enables students to reflect on their work over time, allows them to share their thinking, gives 
them meaningful feedback from their teacher and their peers, and provides teachers with a variety 
of evidence to support their final evaluation (McGuire, 2005). 

Assessment in education using web tools, also known as e-assessment, has been the subject of 
several studies (Pundak, Maharshak & Rozner, 2004; McGuire, 2005; Nicol, 2009; Pachler, Daly, 
Mor & Mellar, 2010; Stodberg, 2012). Gikandi. Morrow and Davis (2011) presented a review of 
the literature related to effective formative e-assessment, and emphasized the role of feedback to 
both students and to teachers. Many studies have focused on e-assessment in science education, 
especially in higher education (Cole & Todd, 2003; Hauk & Segalla, 2005; Nirmalakhandan, 
2007), while others have focused on the subject of physics (Mestre, Hart, Rath & Dufresne, 2002; 
Martin-Bias & Serrano-Fernandez, 2009). Physics, as a special case in upper level science 
education, is considered challenging for both students and teachers (Angell, Guttersrud, 
Henriksen & Isnes, 2004; Chandra & Watters, 2012). Online quizzes are used as a means of e- 
assessment in several design approaches. In some studies, they consist mainly of multiple-choice 
or true-false questions to test conceptual understanding (Martin-Bias & Serrano-Fernandez, 
2009). Sometimes quizzes contain problems, and are mainly used as a substitute for written 
homework, often allowing repetition (Bonham, Deardorff & Beichner, 2003; Cole & Todd, 2003; 
Hauk & Segalla, 2005; Liberatore, 2011). In these studies, the online homework was found to 
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have a negligible to moderate advantage over written homework in terms of course outcomes, 
while student attitudes were seen to be generally positive. Kortemeyer, Kashy, Benenson and 
Bauer (2008) have expressed concern that multiple attempts at online homework might lead 
students to adopt a trial-and-error approach. 

In this study, we investigated the role of online Moodle quizzes in the learning environment of a 
blended learning model (based on the stages presented in Figure 1). We posed the following main 
research questions: 

• Are the online quiz grades related to other methods of assessment (a final written exam 
and written tests)? 

• What are student behaviors during the online quizzes? 

° How many attempts did the students actually make on the multiple-attempts online 
quiz? 

° Was there a noticeable change in successive attempts (grades and solving time)? 

• What are the students' attitudes towards the online quizzes? 

° To what extent did the students use symbol manipulation when solving problems on 
the online quizzes? 

° How many attempts should be allowed, according to student perceptions? 


2. Methods 

2.1. Context of the study 

The main goal of this research was to study the learning outcomes and attitudes of students with 
regard to online Moodle quizzes in order to improve instructional design in an introductory 
physics course for life sciences at Tel-Hai College, in the Upper Galilee region of northern Israel. 
The physics course (algebra-based mechanics) was taught by the first author, according to a 
blended learning model. It was based on large, traditional face-to-face lectures, practice sessions 
in smaller groups of about 25 students and a rich Moodle learning environment. The lectures 
focused on conceptual understanding of mechanics, while the practice sessions were used to 
develop problem-solving skills. The Moodle environment that accompanied the course included 
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some attractive features both for students and instructors. The course site was built with several 
supportive tools for learning: PowerPoint presentations, lecture videos, links to java animations 
that were demonstrated during classes, links to other physics sites, such as short lectures in TED 
format, a weekly set of problems to be used for self-assessment, full or partial solutions to these 
problems, another weekly set of problems that were worked out during practice sessions with 
tutors, a discussion forum and four quizzes that were used for formative assessment, together 
with written tests. In each quiz, we included some conceptual questions, often as multiple choice 
questions, as well as regular physics problems, where an unknown numerical value was required 
as the answer. 

The students' knowledge and understanding were assessed weekly throughout the semester, using 
two different methods: ordinary, short written tests restricted to 20 minutes were given during the 
second, fourth and tenth weeks of the semester. In four other weeks, the students were required 
to answer an online quiz in a Moodle environment. The online quizzes were open for one week, 
and students could choose where and when to take them. Once a student started a quiz, there 
was a mandatory limit of one hour prior to submission. For the first three quizzes, the students 
did not receive their grade until the end of the week, and were allowed only one attempt. These 
online quizzes were a substitute for the written tests, and were similar to them in that the 
feedback was given after a few days, and only one attempt was allowed. The last quiz was open, 
with five attempts allowed. After each attempt, limited, but immediate feedback was given, 
indicating the overall score and which questions were answered correctly. The quizzes had a 
randomization factor permitted by the Moodle environment, so that in the next attempt, the 
student was required to answer the same questions as in the previous attempt, but with different 
numeric results. Fuller and richer feedback, including verbal feedback and clues about how the 
questions could be answered, was given at the end of the week. 
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2.2. Research instruments 

The first stage in this research was based on the analysis of student grades. Two instruments were 
used in the second and the third stages in the study: 

• Data from log files that were retrieved from Moodle, to examine the students' behavior: 
how many attempts they made, whether there was an improvement between the first and 
last attempts, how much time the students spent on a single attempt, whether there was a 
change from the first attempt to the last, and whether there was any correlation between 
the quiz outcomes and the score on the final exam. 

• A questionnaire distributed towards the end of the semester to assess the students' 
attitudes towards the online quizzes, as compared to the written tests. The questionnaire 
consisted of four sections. The first included background information, specifically gender 
and department of study. The second comprised 14 statements that were divided into 
four categories: feelings when conducting the online quiz, attitudes toward the quiz and 
the preparation required for success, contributions to self-assessment and the use of 
symbol manipulation when solving the problems on the quiz (see the next paragraph). 
The students were required to rank each statement in each category on a scale of 1 to 5: 
5. “strongly agree,” 4. “agree,” 3. “undecided,” 2. “disagree,” 1. “strongly disagree.” The 
Cronbach alpha reliability coefficient for this section was a = 0.887. We calculated the 
means and standard deviations for the scores in each category. In the third section, the 
students were asked whether they preferred online quizzes or written tests, and were 
required to explain their choice in their own words, and explicitly relating to the 
contribution they made to their learning and understanding. The fourth section was 
similar to the third, but was related to the number of attempts permitted on the online 
quiz, as recommended by the students. 

We were particularly interested in assessing the use of algebraic symbol manipulation while 
solving a problem on the quiz. The quiz consisted of a few physics questions, in which a system 
is described, sometimes with a picture, and a specific physical entity is to be calculated. The 
students had to determine which physics laws applied to the described system, write the 
appropriate equation based on the law, and calculate the missing entity from the equation, using 
algebraic manipulations. Most of this process is not gauged by the learning environment, as only 
the numeric final answer is tested. Many students find it easier (Simon & Simon, 1978) to 
manipulate numbers instead of symbols, while experts manipulate symbols and only at the end 
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substitute the symbols with the given numbers. The latter strategy has an advantage over a quiz 
that allows several attempts, such as the last online quiz. This is because the questions in the 
following attempts were the same, so that the laws and equations applied were also the same, but 
the numbers, and therefore the final numeric answers, were different. Using the symbol 
manipulation strategy could save a lot of time for questions that were answered correctly in the 
earlier attempt, but had to be answered again with different numbers in the following attempts. 
We wanted to determine whether the students were aware of this advantage, and if they tended 
to use this strategy. 


2.3. Study participants 

The analysis in the first stage in this study (as presented in Figure 1) was based on the scores of 
173 students from the life sciences department (56% males, 44% females) who participated in the 
course. The second and the third stages were based on 120 students who answered the attitudes 
questionnaire that was administered towards the end of the semester. Sixty-six of them (55%) 
were male and 54 (45%) were female. The students came from four different departments: 
Biotechnology (37%), Animal Sciences (10%), Environmental Sciences (24%), and Food Sciences 
(23%). Eight students did not mention their department. It is important to emphasize that there 
is a less than absolute overlap among the participants in the different stages. 


2.4. Data Analysis 

The main purpose of the online quizzes was to serve as formative assessment for students, i.e., to 
aid students in understanding the course material. In order to determine whether our 
expectations were fulfilled, we analyzed the data retrieved from the Moodle log files, as the 
Moodle environment keeps track of every activity by every student in the environment. Some of 
the information we looked for was straightforward. For example, the quiz grades were exported 
to an excel file and later added to the SPSS tool to be statistically processed and compared to 
other features, such as the final grades, by means of a linear regression. Other types of 
information required programming tools for their retrieval. For example, the number of attempts 
each student actually submitted had to be extracted from a file containing the scores for every 
attempt. As there were more than 400 attempts — meaning more than 400 lines in the file, we 
used programming to extract this information. Although data mining tools do exist to deal with 


- 196 - 



Journal of Technology and Science Education — http://dx.doi.org/10.3926/jotse.217 


such problems (see for example Romero et al., 2008; Siemens & Long, 2011), we found it easier 
to write a simple program in the C programming language by ourselves. We used linear regression 
to determine whether the grades on the online quizzes predicted the grade on the final exam of 
the course. We examined other possible predictors of the final grade, such as gender, grades on 
written tests, the number of attempts on the last quiz, time spent taking the quizzes, and the 
attitudes of students toward the online quizzes. An important part of formative assessment is the 
ongoing improvement gained from the repetitive nature of solving a problem, receiving 
immediate feedback, reflecting, and trying again. To determine whether the students met this 
challenge, we looked for indications of improvement from the first attempt to the last. We 
examined two indicators - the grade on the attempted quiz and the time spent trying to complete 
the quiz during each specific attempt; to do so, we retrieved these data for the first and last 
attempts at the last online quiz, for every student. Note that the last attempt was the same as the 
first attempt for students who took the quiz only once. For each indicator, we compared the first 
and last attempts using a t-test. All statistical analyses were conducted using SPSS software. 


3. Results 

In the first stage of the data analysis, we investigated the predictors of success on the final course 
exam. We used a linear regression to test four predictors: average grade on written tests, average 
grade on online quizzes, gender and science department. Results indicate that gender was not a 
significant predictor in the model (p>0.05). The science department was found to be atavistically 
significant (F = 3.92, P=0.050), but the explained variance value (r 2 ) was low (about only 4%), 
and therefore we did not include it in the model. Both the average grade on written tests and the 
average grade on online quizzes were found to be significant predictors of the grade on the final 
exam, as shown in Table 1. High scores on the written tests during the course, as well as on the 
online quizzes, predicted significantly high scores on the final written exam. 


Predictor 

Significance 

Explained 
variance value r 2 

Regression equation 

Standardized 

equation 

Average grade on 
written tests 

F=40.036 

p=0.000 

29% 

y=37.520 + 0.419x 

Zy=0.541*zx 

Average grade on 
online quizzes 

F=24.677 

P<0.003 

20% 

y=23.264 + 0.521x 

Zy=0.450*zx 


Table 1. Regression models for predictors of the final grade 
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After identifying the role played by the online quizzes in the evaluation process, we examined 
student behaviors as they completed the quiz. The following results compared the first and last 
attempts on the last online quiz. As noted, the last attempt was the same as the first for students 
who attempted to complete the quiz only once. There was a significant difference in both the 
time taken to complete the quiz and the grade achieved (both with large effect size [Cohen’s d]), 
suggesting an improvement in students' performance. Students significantly improved their 
scores (Meanfirst attempt=53.38, S.D=31.12, Meanlast attempt =82.39, S.D=22.99, t=-10.7, 
P<0.001, Cohen’s d= 1.380) and greatly shortened their performance time (Meanfirst 
attempt=38.85, S.D=15.26, Meanlast attempt =26.13, S.D=13.49, t=-7.09, P<0.001, Cohen’s 
d=0.915). The results are presented in Figures 2 and 3. 
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Figure 2. First attempt vs. last attempt on the last online quiz — Student grades 
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First attempt 


Last attempt 


Figure 3. First attempt vs. last attempt on the last online quiz — Student response time 


3.1. Student attitudes 

Close inspection of the students’ attitudes towards online quizzes reveals a generally positive 
attitude. Table 2 presents the results. 


Category 

Cronbach’s a 

Mean 

Min=l, Max=5 

S.D. 

Example of statement 

Feelings 

0.89 

3.36 

1.02 

"I feel relaxed when I complete the 
online quiz" 

Behaviors 

0.81 

4.06 

0.79 

"In preparation for an online quiz, I 
do exercises, just like I prepare for a 
written test" 

Contribution to self- 

assessment 

0.89 

3.44 

0.88 

"Completing the quiz helped me 
figure out my problems with the 
course material" 

Use of symbol 
manipulation 

0.83 

3.35 

1.01 

"I use symbol manipulation when I 
complete the quiz" 


Table 2. Student attitudes toward online quizzes, organized by category 


No differences were found with respect to gender. A positive, statistically significant relationship 
was found between students' attitudes and their scores on online quizzes, but the explained 
variance value (r 2 ) was low (F=16.065, p=0.000, r 2 =13%, y = 41.938+ 8.537x, Zy = 0.361*zx). 

No significant relation (as tested by Pearson tests) was found between the level of symbol 
manipulation use (as reported by students on the questionnaire) and the number of attempts or 
the time taken to complete the quiz (as measured by the Moodle system). 
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In answer to the summary question: "If you had the choice, which would you prefer — an online 
quiz or a written test?" 76% chose online quizzes. In the open-ended question that followed, 
asking for an explanation for the above answer, some students (7) commented that they would 
prefer a combination of online quizzes and written tests, as was actually given during the 
semester. We have left the analysis of other open-ended answers for a later study. 

In order to answer research questions 3 to 6, we analyzed both the relevant question on the 
questionnaire and students' performance in the Moodle system. 


3.2. Recommended vs. actual number of attempts 

The question "How many attempts at an online quiz would you recommend?" was answered by 
112 students, with the following frequency: 4 (3.57%) recommended only one attempt, 39 
(34.82%) recommended 2 attempts, 45 (40.18%) recommended 3 attempts, 14 (12.50%) 
recommended 4 attempts and 10 (8.93%) recommended 5 attempts for each quiz (Figure 1). It is 
interesting to compare these results with the actual number of attempts that the students 
submitted for the last quiz. One hundred and seventy-three students submitted 418 attempts at 
the last online quiz, with the following frequency: 49 (28.32%) submitted one attempt, 52 
(30.06%) submitted 2 attempts, 36 (20.81%) submitted 3 attempts, 23 (13.29%) submitted 4 
attempts and 13 (7.51%) submitted 5 attempts. The most prominent difference between actual 
and recommended frequencies is related to one attempt. While a negligible number of students 
recommended 1 attempt, more than 28% actually performed only one attempt. The results are 
presented in Figure 4. 



Figure 4. Frequency of recommended and actual number of attempts on an online quiz 
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4. Discussion 

Our research focused on online quizzes as a learning object and investigated their role in 
instructional design. Wiley (2000) and Lehman (2007) defined a ‘learning object’ as a reusable 
digital component that can be selectively applied alone or in combination by computer software, 
learning facilitators, or the learners themselves to meet individual needs for learning or support 
their performance. The online quizzes in our instructional design serve as a practice and review 
component (one of the three types identified by Shepherd in 2006), with an emphasis on the 
formative assessment process. 

The online quizzes provide information that yields feedback for making necessary modifications 
to improve teaching and learning. We focused on promoting a better instructional design for a 
physics course in higher education, demonstrating formative assessment as Daly, Pachler, Mor 
and Mellar (2010) have proposed. They suggested five strategies for formative assessment: 

• engineering effective learning tasks that elicit evidence of learning; 

• providing feedback that moves learners forward; 

• clarifying, understanding and sharing learning intentions and criteria for success; 

• activating students as owners of their own learning; and 

• activating students as instructional resources for one another. 

We will now discuss the main results found in our study with respect to our design approach to 
online quizzes. We found a significant correlation between quiz grades (written and online) and 
final exam grades. This finding increases the sense of reliability when it comes to integrating 
computerized assessment tools in higher education as part of the students' assessment process. 
These computerized tools allow students to flexibly choose the time and place to act. No 
significant difference was found in relation to gender, in contrast to what was found in studies by 
Kortemeyer (2009) and Sanders and Morrison-Shetlar (2001). 

No significant relationship was found between the level of use of symbol manipulation and the 
number of attempts or the time taken to complete the quiz. This might indicate that the 
development of generalization in problem solving requires explicit intervention by instructors. 
Several attempts offer the advantage of using symbol manipulation, because the questions in the 
following attempts were the same. Thus, the laws and equations applied were the same, but the 
numbers, and therefore the final numeric answers, were different. The results indicate that most 
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of the students were probably not aware of this advantage and they did not tend to use this 
strategy. The results strengthen Simon and Simon’s (1978) claim that, unlike experts, students find 
it easier to manipulate numbers as opposed to symbols during problem solving. The results 
focused attention on the students’ mathematical thinking and allowed instructors to improve 
learning and teaching process. 

We found important differences between the number of attempts the students recommended to 
be allowed and the actual number of attempts they made. It is interesting to focus on the 
restriction to one attempt. According to this approach, the quiz is used as a test that evaluates 
knowledge and skills, but does not allow for correcting the wrong answers. About 72% of the 
students attempted the quiz more than once, and significantly improved their grades. This can be 
explained by the fact that students do better when they feel confident (Pintrich & De Groot, 
1990). Knowing that there are more attempts available gives them the confidence they need to 
perform better. However, that does not necessarily mean that they intend to use the available 
attempts to improve learning and try to achieve a better grade. This last behavior is related to 
responsibility, i.e., the willingness to take responsibility for one's learning. Only 4% of the 
students recommended that the quizzes be given with one attempt, while 28% of them actually 
attempted the quiz only once. Most of the students in this study recognized the potential 
benefits, took responsibility and attempted to achieve a better grade. It is useful to note that no 
significant difference was found between the average grade on the quiz and the number of actual 
attempts. This finding suggests that there are some students who were satisfied with their low 
grade, or assumed that another attempt would not help them achieve a better grade and therefore 
did not make any effort to improve it. Instructors can use this tool to identify these students and 
offer them the help they need to learn better by using alternative teaching methods. Notably, only 
21% of the students who answered the questionnaire recommended more than three attempts at 
the quiz. A possible explanation for this is that the students were aware of the phenomenon that 
Kortemeyer et al. (2008) referred to as "turning thinkers into guessers", and they therefore 
preferred to have a small number of allowed attempts. It seems that students understand the 
usefulness of the suggested design approach, and do not want to be tempted by many attempts 
that can lead them into a trial-and-error cycle, rather than guiding them towards a better 
understanding. 

Other useful results found in this study are in line with the literature. These include the positive 
attitudes of students towards online quizzes and the relevance of the online quizzes to student 
achievement (Sanders & Morrison-Shedar, 2001; Liberatore, 2011). Furthermore, we found an 
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improvement in the students' performance throughout the learning process, from the first 
attempt at completing the quiz to the last attempt. The mean grade improved and the time spent 
completing the quiz decreased. This reflects the willingness of the students to enter into the 
meaningful learning process offered by the specific course design. 

Educational institutions are increasingly looking for effective and productive technological aids 
for the development of new models of teaching and learning. A wide range of design options are 
offered, ranging from a traditional design that includes face-to-face meeting, to hybrid or blended 
models and e-learning courses. Our research emphasizes the role of online quizzes as a tool for 
formative assessment that enables greater interaction between the teachers and their students, 
and allows learners to make decisions concerning their studies and build their knowledge based 
on their experiences. Nevertheless, efforts are still needed in this area. The physics course design 
presented some formative components that were investigated, for the first time, in our research. 
Continued research will lead to further development in the area of course design that will also 
consider other formative components, such as skill performance tools. In order to improve 
course design, we suggest using further qualitative research tools, such as interviews, in addition 
to those that were used in this study. This will provide greater insight into how students feel 
(resistance or support) about the course components. 

In this article, we describe the successful implementation of online Moodle quizzes in a first-year 
introductory, algebra-based physics course for the life sciences. One quiz was given with five 
allowed, time-restricted attempts, with feedback following each attempt. This design approach 
using online quizzes is slightly different from the conventional design approach described in 
many articles. In the conventional design approach, online quizzes are used as online homework, 
with a due date, but there is no time restriction for an individual attempt. We believe that this 
difference is important, because it changes the way the student approaches the quiz. Using our 
approach, the student has to prepare for the quiz, cover the relevant topics, and do exercises 
before attempting to take the quiz. 
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